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This paper describes some techniques for efficient coding of two-tone 
(black and white) facsimile pictures. These techniques use the two-dimen- 
sional correlation present in spatially close picture elements to change the 
relative order of transmission of elements in a scan line. This ordering 
increases the average length of the runs of consecutive black or white ele- 
ments in the ordered line, making the data more amenable to one-dimen- 
sional run-length coding. We describe several variations of the ordering 
scheme, which differ in complexity and coding efficiency and evaluate 
their coding efficiency. For a variety of 8-1/2 inch by 11-inch typed docu- 
ments, road maps, and circuit diagrams scanned with 200 lines /inch, 
these techniques reduce the bit rate by 30 to 50 percent over and above the 
one-dimensional run-length coding along a scan line; for single-spaced 
typed material with 100 lines/inch, this reduction is about 25 percent. 
We compare one of our techniques with a two-dimensional compression 
technique recently proposed by Preuss. We show that our technique results 
in an entropy about 10 to 18 percent lower than that obtainable through 
Preuss' technique. 

I. INTRODUCTION 

Transmission and/or storage of two-tone (black and white) pictures, 
such as weathermaps, printed texts, etc., have been receiving con- 
siderable attention for some time. The practical importance of this 
problem is evidenced by the number of facsimile communication 
systems that are now available. 1 As the cost of electronics decreases 
faster than transmission costs, it is becoming advantageous to use 
sophisticated facsimile terminals to reduce transmission costs and time, 
and, indeed, many of the recent facsimile communication systems have 
resorted to various source encoding techniques to utilize the statistical 
redundancy between the spatially close picture elements to reduce 
the bit rate required for transmission. 2-5 

The picture elements along a scan line of a facsimile picture consist 
of runs of white picture elements (pels) separated by runs of black 
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picture elements. Values of the spatially close picture elements are 
significantly correlated. Source coding techniques, which do not reduce 
the "information content" of the pictures (i.e., it is possible to con- 
struct the original picture exactly without any degradation from the 
coded picture) use the statistical redundancy either along a single 
scan line or along many scan lines. One-dimensional run-length coding 
techniques 6 code the runs of black or white elements along a scan line. 
Development and performance of many different codes to code the 
runs have been a subject of many papers. 7-9 Some of these codes are 
capable of performing close to the entropy of the run-length statistics. 
Extensions of this basic run-length coding scheme have been made to 
include line-to-line correlations. 10-12 Two-dimensional correlations 
have also been used in development of block coding 13 as well as blob 
coding 14 methods. 

In this paper, we describe techniques which use the two-dimensional 
correlation of the picture signals. Specifically, our techniques consist 
of changing the relative order of transmission of the picture elements 
along a scan line in such a way as to increase the average run length 
of the black and/or white elements. A reference signal is constructed 
from the previously transmitted data, and the data in the present 
line is ordered with respect to this reference signal. A memory is used 
to store the incoming bits of new data in a manner such that the 
address for storing a particular input bit is derived from the reference 
signal. At the end of the ordering period, the information stored in 
the memory is read out in a sequential manner and run-length coded. 
The receiver decodes the run-length coded information and stores it 
for a given ordering interval. The original data stream is then re- 
constructed by reading out the stored data in the same order in which 
it was stored at the transmitter. Our techniques can be classified into 
three broad categories. In the first category, described in Section 2.1, 
we order a line of picture data using the previously transmitted line 
immediately above it. Thus, the elements of the previous line are 
taken as the reference signal for ordering. In the second category, 
described in Section 2.2, we predict an element of a line from the value 
of a corresponding element from the previously transmitted line, and 
order the prediction error of the present line using elements of the 
previous line as the reference signal. The third category, which is 
described in Section 2.3, uses several already transmitted elements both 
from the present line and the previous line to define a state. We 
develop a predictor as a function of the state, as one which minimizes 
the prediction error conditioned on that state. We then sort the states 
in terms of probability of the prediction error associated with each 
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state, and order the prediction error (using the state-dependent 
predictor) according to the goodness of the state. During ordering, 
prediction errors corresponding to "good" (states with low probability 
of prediction error) and "bad" states are arranged in a sequence in two 
different parts of a memory. Contents of the memory are then read 
out and run-length coded for transmission. Since prediction errors 
corresponding to "good" states are mostly 0, this technique increases 
the length of the runs and consequently achieves bit-rate reduction. 

1.1 Summary of results 

Our simulations indicate that, for pictures with 200 lines/inch, the 
previous-line-ordering technique reduces the entropy by about 20 
percent over the entropy using one-dimensional run-length coding. 
Using previous-line-element-prediction and previous-line-ordering, 
this reduction is about 30 percent. State-dependent prediction and 
ordering reduce the entropy by about 33 to 50 percent. This reduction 
is about 25 percent for a picture with 100 lines/inch. Our results 
indicate that our state-dependent predictor does not vary significantly 
with pictures and, therefore, may not have to be transmitted for most 
pictures. Also, most of the advantage in using ordering based on 
"good-bad" state-groups is obtained by using only two state-groups. 
Among the algorithms that we compare ours with, is an algorithm 
recently proposed by Preuss. 12 We show that our algorithm is about 
10 to 18 percent more efficient in terms of entropy. 

II. CODING ALGORITHMS 

In this section, we describe each of our coding algorithms in detail 
and present results of our simulations on the computer. The computer 
simulations were done on pictures with 256 lines and 256 elements per 
line. The resolution was either 200 lines/inch or 100 lines/inch. The 
pictures we used included a drawing of a schematic, a map, and the 
inside part of text material (both single- and double-spaced typing). 
Sections of pictures used are shown in Fig. 1. Figures la and lb are 
sections of single-spaced text with 200 lines/inch and 100 lines/inch, 
respectively. Figure lc is double-spaced text with 200 lines/inch and 
Figure Id is part of a circuit diagram. In addition to these, we used 
a map which is a section of page 19 from Ref. 15. As a measure of 
performance, we used the entropy of run-length statistics. We com- 
puted the entropy of black and white runs and the average black and 
white run lengths. Using these and eq. (1), we computed the entropy 
in bits/pel (assuming that the number of black and white runs are 
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Fig. 1 — Sections of figures viewed through a television camera and converted to a 
two-tone picture by a simple thresholding technique. Figures (a), (c), and (d) had 200 
lines/inch and an array of 256 X 256 was used for simulation. Figure (b) was taken 
at 100 lines/inch. 
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where 

H w is the entropy of the white run statistic (bits/run), 
Hb is the entropy of the black run statistic (bits/run), 
r w is the average white run length (pels/run), 
n is the average black run length (pels/run), 
H is the entropy in bits/pel. 

2.1 Ordering present line with reference to previous line 

This technique orders the present line with respect to the previous 
line. Consider a memory containing 256 cells (equal to the number of 
elements per line). We store in this memory elements from the present 
line. Assume for the sake of explanation that the memory is arranged 
along a line and memory location 1 corresponds to the left-hand 
side and location 256 corresponds to the right-hand side of the memory. 
If the first element of the previous line is white (=1), then we put 
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the content of the first element of the present line on the left-hand 
side of the memory. If, on the other hand, the previous line element 
is black ( = 0) , we put the first element of the present line on the right- 
hand side of the memory. We then put the second element on the 
right-hand or left-hand side of the memory depending on whether 
the second element of the previous line is black or white. This process 
is continued until the entire present line is ordered and the memory 
is filled. The information stored in the memory is coded as runs of 
black and white elements. It is easy to see that the present line can 
be uniquely reconstructed from the knowledge of the run lengths of 
the ordered line since the ordering information is known to the receiver. 
The entropies obtained using the ordered run-length statistics are 
given in Table I. This table also shows, for comparison purposes, the 
entropy of the picture using the statistics of simple one-dimensional 
(along a scan line) unordered run lengths. The ordered entropy of the 
run lengths varies between 0.12 bit/pel to 0.24 bit/pel for 200 lines/inch 
resolution pictures. The increase in coding efficiency ( = decrease in 
entropy) due to ordering over plain run-length coding is of the order 
of 20 to 25 percent. This increase in efficiency is decreased to 16 
percent for the picture with 100 lines/inch. In Fig. 2, we show the 
original picture (same as Fig. la) and its ordered version. It is interest- 
ing to note that the picture elements on the left side of the ordered 
picture are mostly white and those on the right side of the picture are 

Table I — Entropy comparisons for different coding algorithms 



Coding Technique 


Entropy (bits/pel) 
Picture 


1 


2 


3 


4 


1 at 100 
lines/in. 


(1) One-dimensional run-length coding 

(2) Present line ordered with reference to 

previous line 

(3) Finite length ordering (length = 64) 

of Technique (2) (length = 128) 

(4) Run-length coding of prediction error 

using previous line predictor 

(5) Technique (4) with ordering using 

previous line 

(6) Run-length coding of prediction error 

using state-predictor 

(7) Technique (6) with state-ordering using 

2 state-groups 
4 state-groups* 
16 state-groups* 
Technique of Preuss using 2 state-groups* 


0.30 

0.24 
0.29 
0.28 

0.25 

0.21 

0.24 

0.20 
0.20 
0.19 
0.22 


0.16 

0.12 
0.17 
0.15 

0.13 

0.11 

0.13 

0.10 
0.10 
0.099 
0.11 


0.21 
0.16 

0.15 

0.14 

0.15 

0.11 
0.11 
0.11 
0.13 


0.23 
0.17 

0.16 

0.15 

0.16 

0.12 
0.12 
0.119 
0.14 


0.38 
0.32 

0.35 

0.31 

0.34 

0.30 
0.30 
0.29 
0.34 



* These entropy numbers do not include extra bits required to specify number of 
elements in each state-group. 
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lb) 
Fig. 2 — (a) Picture of original, (b) Present line ordered with respect to previous line. 

mostly black. If the vertical correlation between the scan lines was 
perfect, there would be no scattered black and white elements ; all the 
white elements would be to the left and all the black elements would be 
to the right. The increase of coding efficiency is intuitively obvious by 
comparing both the original and the ordered pictures of Fig. 2. 

2.1.1 Finite length ordering 

To evaluate the effects on coding efficiency of ordering only a part 
of the line, we simulated finite length ordering. This has the advantage 
to some extent of localizing along the horizontal direction the effect of 
transmission errors. However, vertical propagation of transmission 
errors is still possible. To illustrate this scheme, consider two memories 
of 128 cells each (half the number of samples/line). We then order the 
elements of the present line as before for each half of the line. However, 
in the first memory, we put elements corresponding to black elements 
of previous line to the left side and elements corresponding to the 
white elements of the previous line to the right side ; whereas, for the 
second memory we reverse the sides for the black and white elements, 
i.e., elements of the present line corresponding to the black elements 
of the previous line are put on the left side. This minimizes the effects 
of "discontinuity" at the boundary of the two memories. The two 
memories are now arranged back to back, and their contents are read 
out sequentially and are run-length coded. The results of simulating 
this scheme are shown in Table I for different sizes of the memories. 
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The increase in the entropy by dividing the line into two segments is 
significant. Thus, there is a considerable loss in coding efficiency due to 
finite length ordering. This allows us to conclude that by ordering 
two lines instead of parts of a line and arranging them back to back, 
as above, there may be further improvement in coding efficiency. 

2.2 Ordering present line prediction errors with reference to previous line 

This coding technique is similar to the one described in Section 2.1, 
except that now we order the prediction errors of the present line. 
We take the predictor to be the corresponding picture element in the 
previous line. We now order this prediction error as before; i.e., if the 
previous line has a white element, we put the prediction error of the 
present line to the left side of the memory and vice versa. As in 
the previous technique, from the ordered line, it is easy to decode 
uniquely the contents of the present line. 

The entropy of the pictures using the run lengths of the ordered 
prediction error is between 0.11 bit/pel and 0.21 bit/pel for pictures 
with 200 lines/inch, as seen from Table I, and this amounts to a 30 to 
35 percent decrease in entropy over simple one-dimensional run-length 
coding. To calculate the advantage of ordering, we also measured the 
entropy of the picture using the run-length statistics of the prediction 
error with the previous line element as, predictor. The entropy of this, 
which is shown in Table I, varied between 0.13 bit /pel to 0.25 bit /pel, 
for pictures with 200 lines/inch. Thus, the reduction in entropy due 
to ordering was 7 to 16 percent over and above the entropy of the 
prediction errors. For the picture with 100 lines/inch, ordering the 
prediction errors brought the entropy down to 0.31 bit/pel, which is a 
reduction of about 17 percent over the entropy of one-dimensional 
run-length coding. 

2.3 State-dependent prediction and ordering 

The technique described in this section differs from the two earlier 
techniques in its use of more picture elements spatially close to the 
present element. It uses these elements to define a state of the present 
picture element. We develop a predictor for each state and use the 
state also to order the present line. Using this technique, it is possible 
to separate the process of prediction and ordering. 

To illustrate the technique, consider the picture element configura- 
tion shown in Fig. 3. The state of the present picture element X is 
defined by elements A, B, C, and D. Thus, state Z is the four-tuple 

Z = (A, B, C, D). (2) 

Since each of the elements A,B,C, and D can have two possible values, 
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Fig. 3 — Configuration for state definition. 

there are 16 states, which we denote by the set [Zt], i = 1, • • •, 16. 
The development of a state-dependent predictor is our next task. 
Such a development is as old as predictive coding itself. 12,16 The 
predictor is developed by the following criterion. We first compute 
P(X = 'Black' | Z = Zi), the probability of the present picture element 
X being Black given the state Z = Zi. The predictor C(Z») for a 
given state Zi, is then, 

C(Zi) = 'Black' if P(X = 'Black' \Z = Zi) > 0.5 

= 'White' otherwise. (3) 

It is easy to see that this predictor minimizes the probability of 
making an error given that a particular state has occurred. We have 
calculated the predictor for each state using several pictures. These 
are shown in Table II. For most states, the predictors do not depend 
upon the picture used, except for a few states that are marked with 
asterisks in Table II ; thus, it is not necessary to transmit the predictors 
for each different picture. We shall evaluate the effects of using the 
predictors of one picture for other pictures. 

Having developed the predictor, we sort the states into two groups. 
The probability of correct prediction using the state-dependent 
predictor is shown in Table II. We note that the probability of correct 
prediction is always higher than 0.5 due to our predictor being a 
minimum prediction error predictor. States which have high proba- 
bility of correct prediction will be called "good" states. Our ordering 
strategy depends upon the goodness of the state. Let the 16 states be 
divided into two groups: one containing "good" states and one con- 
taining "bad" states. Our state-dependent ordering algorithm then 
works as follows : we first evaluate the prediction error for a particular 
picture element in the present line by using the state-dependent 
predictor. Then, if the state is a "good" state, we put the prediction 
error on the left side of the memory, and if the state is "bad", we put 
the prediction error on the right side of the memory. In all our simu- 
lations, we used a threshold of 0.8 for the probability of prediction 
error to determine the goodness of a state. Having ordered the predic- 
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tion error, we then code the run lengths of the prediction error. It is 
easy to see that the line of data can be uniquely constructed from the 
coded run lengths of the prediction error. The entropy of run lengths 
of such ordered prediction errors for different pictures is given in 
Table I. For pictures with 200 lines/inch resolution, the entropy 
varies between 0.10 bit/pel to 0.20 bit /pel. This represents a decrease 
of between 33 percent and 49 percent over the entropy of simple 
one-dimensional run-length coding. We evaluated, for comparison 
purposes, the entropy of the picture using the run-length statistics 
of the prediction error. This varies between 0.13 bit/pel to 0.24 bit /pel 
for the pictures with 200 lines/inch. It is clear then that state-dependent 
ordering allows us to decrease the entropy by about 16 to 25 percent 
over and above the entropy obtained by using the prediction error 
of the state-dependent predictor. 

We described a scheme in which only two groups of states were used 
for ordering purposes. To evaluate the effect of using more than two 
groups, we divided the states into 16 groups, and ordered the pre- 
diction error as before. In the case of more than two groups of 
states, it is not possible to decode the original line of picture data from 
coded run lengths unless extra information about the number of 
elements in each state-group is specified for each line. The entropy of 
the run-length statistic using more than two state-dependent groups 
is shown in Table I. These figures of entropy do not include the extra 
information that is required to be transmitted about the number of 
elements in the group. The increase of coding efficiency by using more 
than two groups is somewhat small and it would be offset completely 
by the extra information mentioned. Thus, most of the decrease of 
entropy due to state-dependent ordering is obtained by using only 
two groups. 

In all of the state-dependent coding algorithms, we have not made 
an effort to optimize several of the coder parameters. Thus, for example, 
when the state-groups are less than 16, we could optimize the groupings 
of the states. The groupings that we used were intuitive and somewhat 
ad hoc. We did vary the groupings in the case of two groups and found 
that the entropies did not change significantly. It appears that optimi- 
zation of groupings may not result in any significant entropy reduction. 

2.3.1 Sensitivity to picture variation 

Picture content affects three parameters of our state-dependent 
coding algorithms : the state-dependent predictor, the entropy numbers 
which depend on the statistics of the run lengths, and the definition 
of "good" and "bad" (or the groupings) states. We studied the sensi- 
tivity of our coding algorithms by considering the variation of predic- 
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tors. The second factor mentioned could be studied by developing a 
specific code based on some run-length statistics (of one of the pictures, 
or some "average" picture) and then using it on all the pictures. We 
did not study this aspect of sensitivity of our algorithm. As mentioned 
in the previous section, we studied the variation of the entropy with 
groupings and found that the variation was not too sensitive. Thus, 
it appears that the groupings-information need not be computed or 
transmitted for each picture. 

From Table II, it is clear that for a resolution of 200 lines/inch, the 
predictors are identical for all states for pictures of single-spaced 
typing, double-spaced typing, and schematic. This may be a result 
of our using a predictor based on the local information surrounding the 
picture element. In the case of a map, however, there are two states 
that have a different predictor compared to the first three pictures. 
Both these states were regarded as "bad" states for the coding of the 
map. We used the predictor of the first three pictures for the coding of 
the map ; and using two state groups, we found a 3-percent increase 
in the entropy. Also, in the case of the picture with 100 lines/inch, 
there is only one state (state number 8) which had a different predictor 
than the first three pictures. This was again a "bad" state. For this 
picture, we found the increase in entropy of about 2 percent. This 
allows us to conclude that it may not be necessary to compute and 
transmit the state-dependent predictor information for each picture. 

2.4 Comparison with the algorithm of Preuss 

Most coding algorithms perform differently for different pictures. 
To compare our results with other two-dimensional coding techniques, 
we implemented a coding algorithm proposed by Preuss. 7 This also 
allowed us to bring out certain similarities and dissimilarities between 
our algorithm and that of Preuss. Preuss has developed a state- 
dependent predictor analogous to our predictor. This predictor is 
shown in Table II. It is seen that his predictor differs from our predictor 
for the first three pictures only for state number 13. Also, it differs 
from our predictor for the first picture (100 lines/inch) for state 
numbers 8 and 13. In our simulation of Preuss' scheme, we used the 
predictor tuned to the particular picture rather than Preuss' predictor. 
Preuss computes the prediction errors analogous to our scheme, and 
then codes the run lengths between the prediction errors for each of the 
state-groups separately, using a different run-length code for each 
state-group. We, on the other hand, use the state-groups to order 
the present line and encode the run lengths of the entire ordered line 
of the prediction errors. Preuss has to specify the number of elements 
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in each state-group;* for K state groups with JV elements in each line, 
this may amount to (K — 1) log2 N extra bits/pel. In our algorithm, 
we do not need transmission of such information for K = 2. In Preuss' 
scheme, the run lengths have to be terminated at the end of each 
state-group for each line, but in our algorithm, a run may begin in 
one state-group and extend all the way up to the end of the line, 
crossing several state-groups. Despite these disadvantages, we thought 
that Preuss' scheme may result in lower entropy, since his run-length 
code was matched to the run-length statistics of the prediction errors 
corresponding to each state-group. We simulated Preuss' scheme using 
two state-groups that were the same as those used for our algorithm. 
Results of this simulation are given in Table I. Entropy numbers 
given for Preuss' scheme do not include the extra information required 
for the number of elements in each state-group. It is seen from this 
table that, compared to our scheme using two state-groups, Preuss' 
scheme results in a 10- to 18-percent increase in entropy. Thus, our 
scheme appears, at least for the pictures we used, to be more efficient. 

III. DISCUSSION AND SUMMARY 

We have presented three different algorithms for the coding of 
two-tone pictures. All three algorithms are "information" preserving, 
and, therefore, it is possible to decode exactly the original picture with 
no approximations. We have compared our results (only in terms of 
entropy) with some standard algorithms such as : (i) one-dimensional 
run-length coding, (it) run-length coding of the prediction errors using 
several different two-dimensional predictors, and (Hi) a two-dimen- 
sional algorithm of Preuss. We found our algorithm to be 10 to 18 
percent more efficient than the algorithm of Preuss. Admittedly, this 
is not a complete comparison. Other parameters, such as the number of 
samples per line (we used only 256) and varying picture material, may 
offset the comparisons. Also, we did not study many other aspects 
important for a coding system, including the performance in the 
presence of transmission channel errors. 

Our technique can be extended by proper definition of the state to 
the case of two-tone dithered pictures. This will be reported in a future 
paper. 17 
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so that the number of elements in each of the state-groups need not be transmitted. 
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